Ultrasound imaging system and methods of imaging using the same

ABSTRACT

A method of registering the position of an object moving in a target volume in an ultrasound imaging system includes capturing a first ultrasound image of a target volume. A second ultrasound image of the target volume is then captured after the capturing of the first ultrasound image. The position of the object in the target volume is identified using differences detected between the first and second ultrasound images. In another aspect, a region of interest in the target volume is determined. A segment of an operational scan range of a transducer of the ultrasound imaging system encompassing the region of interest is determined. The transducer is focused on the segment of the operational scan range during image capture.

FIELD OF THE INVENTION

The present invention relates generally to imaging systems and,specifically, to an ultrasound imaging system and methods of imagingusing the same.

BACKGROUND OF THE INVENTION

Ultrasound-guided interventional procedures such as breast biopsies andprostate brachytherapy are well-known. Needles can be inserted into thebody and either obtain a biopsy sample or deliver a dose of a selectedtherapy. For biopsies, it is desirable to target a specific volume whenobtaining a tissue sample. Where a dose is being administered to atarget volume, it is desirable to track the precise location of theneedle delivering the dose in real-time to ensure that the therapy isdelivered according to plan.

Radioactive seeds can be used as a therapy to treat tumors in prostates.In order to ensure adequate coverage of the therapy, it is desirable toimplant the seeds a pre-determined distance apart. If the distancebetween the seeds is too large, tissue between the seeds may not receivethe amount of therapy needed for the treatment. If, instead, the seedsare too closely positioned, the tissue can be over-exposed. Further, itis desirable to ensure that the implantation of the seeds is limited tothe target volume in order to prevent the therapy from adverselyaffecting otherwise healthy tissue.

In robotic-aided interventional procedures, such as robot-aided andultrasound-guided prostate brachytherapy as well as free-handultrasound-guided biopsy procedures, a needle is inserted free fromparallel trajectory constraints. Oblique insertion of the needle,however, can result in the needle intersecting the two-dimensional(“2D”) trans-rectal ultrasound (“TRUS”) image and appearing as a point,leading to blind guidance.

Some investigators have developed automatic needle segmentation methodsto locate needles for biopsies and therapy. These methods, however,require that the needle be completely contained in the 2D ultrasound(“US”) image.

The general operation of ultrasound transducers has providedless-than-desirable image resolution in some instances. Image qualityfor less significant regions distal from the target volume or even alongthe shaft of the needles may not be as critical as for the regionsurrounding the needles. This is especially true for therapy where seedsare being implanted in a target volume. Current ultrasound techniques,however, are directed to the capture of generally evenly distributedimages, regardless of the content of the volume targeted by the images.

It is, therefore, an object of the present invention to provide a novelmethod of imaging using an ultrasound imaging system.

SUMMARY OF THE INVENTION

In an aspect of the invention, there is provided a method of registeringthe position of an object moving in a target volume in an ultrasoundimaging system, comprising:

capturing a first ultrasound image of a target volume;

capturing a second ultrasound image of said target volume after saidcapturing of said first ultrasound image; and

identifying the position of said object in said target volume usingdifferences detected between said first and second ultrasound images.

In a particular aspect, a difference map of the differences between thefirst and second ultrasound images is generated. The difference map canbe thresholded to identify significant changes between the first andsecond ultrasound images. In another particular aspect, the object is aneedle, and the difference map is filtered to identify voxels in thedifference map corresponding to a characteristic of the needle. In afurther particular aspect, the first ultrasound image is captured priorto entry of the object in the target volume.

In another aspect of the invention, there is provided an ultrasoundimaging system for registering the position of an object moving in atarget volume, comprising:

a transducer for capturing a first ultrasound image and a secondultrasound image of a target volume; and

a processor for detecting differences between said first and secondultrasound images to identify the position of said object in said targetvolume.

In a particular aspect, the processor generates a difference map fromthe first and second ultrasound images identifying the differencestherebetween. The processor can threshold the difference map to identifysignificant differences between the first and second ultrasound images.

In a further aspect of the invention, there is provided a method ofimaging using an ultrasound imaging system operable to capture imagedata from a target volume, comprising:

determining a region of interest in the target volume;

determining a segment of an operational scan range of a transducer ofsaid ultrasound imaging system encompassing said region of interest; and

focusing said ultrasound imaging system on said segment of saidoperational scan range during image capture.

In a particular aspect, the region of interest is an area of expectedactivity of an object. In another particular aspect, the object is aneedle, and the region of interest includes the area along a trajectoryof the needle beyond a tip of the needle. In a further particularaspect, the determining of the region of interest includes the expectedposition of a needle in the target volume. The transducer can be, forexample, a rotational transducer. In still other particular aspects, thefocusing includes capturing image data in the segment of the operationalscan range at a greater scan density than outside of the segment of theoperational scan range, or capturing image data only in the segment ofthe operational scan range.

In a still further aspect of the invention, there is provided anultrasound imaging system, comprising:

a transducer for capturing ultrasound images of a target volume; and

a processor for determining a region of interest in the target volume,for determining a segment of an operational scan range of saidtransducer encompassing said region of interest, and for directing saidtransducer to focus on said segment of said operational scan range.

In a particular aspect, the processor determines an area of expectedactivity to determine the region of interest. In another particularaspect, the transducer is a rotational transducer and the processordetermines an angular sector of the operational scan range of therotational transducer. In a further particular aspect, the processordirects the transducer to capture image data in the segment of theoperational scan range at a greater scan density than outside of thesegment of the operational scan range. In a still further particularaspect, the processor directs the transducer to capture image data onlyin the segment of the operational scan range.

The invention enables the position of the needle to be accuratelydetermined. By only analyzing image data that varies significantlybetween two ultrasound images, the needle can be readily differentiatedfrom complex backgrounds in the ultrasound images. Further, by focusingon a segment of the operational scan range of the transducer of theultrasound imaging system during image capture, more detailed image datacan be captured around the needle to enable its position to bedetermined with a desired level of accuracy. This can be achievedwithout sacrificing the scanning speed in some cases.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, withreference to the attached Figures, wherein:

FIG. 1 is a schematic diagram of an ultrasound imaging system forimaging a target volume in a subject;

FIG. 2 shows a three-dimensional (“3D”) TRUS transducer forming part ofthe ultrasound imaging system of FIG. 1 capturing a set of 2D US imagesof a needle;

FIG. 3 is a flow chart of the general method of operation of the systemof FIG. 1;

FIG. 4 shows a reconstructed 3D image generated from 2D ultrasoundimages captured by the TRUS transducer shown in FIG. 2;

FIG. 5 is a flow chart illustrating the method of performing asubsequent 3D US scan;

FIG. 6 is a sectional view of a scan range corresponding to a region ofinterest determined using the method of FIG. 5;

FIG. 7 is a flow chart that illustrates the method of segmenting aneedle;

FIG. 8 is a flow chart that illustrates the method of determining thegreyscale-level change threshold;

FIG. 9 is a flow chart that illustrates the method of generating adifference map;

FIGS. 10 a and 10 b show the difference map generated using the methodof FIG. 9 before and after pre-filtration respectively;

FIG. 11 is a flow chart that illustrates the method of performingregression analysis;

FIG. 12 is a flow chart that better illustrates the method of filteringthe difference map;

FIG. 13 shows the difference map of FIGS. 10 a and 10 b immediatelyprior to the performance of the final regression analysis; and

FIGS. 14 a to 14 c show various 2D US images generated using theultrasound imaging system of FIG. 1.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The method of registering the position of an object such as a needleprovides for the near real-time identification, segmentation andtracking of needles. It has a wide range of applications, such as biopsyof the breast and liver and image-guided interventions such asbrachytherapy, cryotherapy, as well as other procedures that require aneedle or needles to be introduced into soft tissues and be positionedaccurately and precisely. The use of the method is described inrobot-aided 3D US-guided prostate brachytherapy for the purpose ofillustration.

Transperineal prostate brachytherapy provides an improved alternativefor minimally-invasive treatment of prostate cancer. Pubic archinterference (“PAI”) with the implant path, however, occurs in manypatients with large prostates and/or a small pelvis. These patientscannot be treated with current brachytherapy using parallel needletrajectories guided by a fixed template, because the anterior and/or theantero-lateral parts of the prostate are blocked by the pubic bone.

To solve the PAI problems, it is desirable to free needle insertionsfrom parallel trajectory constraints. Oblique trajectories allowpatients with PAI to be treated with brachytherapy without firstundergoing lengthy hormonal downsizing therapy. In addition, changes inthe prostate size prior to implantation, where the therapy is determinedin advance of the procedure, and during the implantation, due toswelling of the prostate, may require re-optimization of the dose plan.The combination of precision 3D TRUS imaging, dosimetry and obliqueneedle insertion trajectories can provide the tools needed for dynamicre-optimization of the dose plan during the seed implantation procedureby allowing dynamic adjustments of the needle position to targetpotential “cold spots”. Cold spots are areas more than a desireddistance from seed implantation locations, resulting inless-than-desired exposure. Further, the dosimetry can be dynamicallyadjusted to compensate for deviations in the actual needle trajectoriesor shifting in the target volume.

A 3D TRUS-guided robot-aided prostate brachytherapy system is showngenerally at 20 in FIG. 1. The system 20 includes a TRUS transducer 24coupled to a motor assembly 28 that operates to control the longitudinalmovement and rotation of the TRUS transducer 24. The TRUS transducer 24is also coupled to a conventional ultrasound machine 32 for displayingimage data as it is captured by the TRUS transducer 24. A videoframe-grabber 36 is connected to the ultrasound machine 32 to captureimage data therefrom. The video frame-grabber 36 preferably operates at30 Hz or greater to provide rapidly updated ultrasound images.

A computer 40 is connected to the video frame-grabber 36 and retrievesultrasound images from the memory of the video frame-grabber 36. Thecomputer 40 is coupled to a mover controller module (“MCM”) 44 that iscoupled to and controls the motor assembly 28. The computer 40 is alsoconnected to the TRUS transducer 24. Further, the computer 40 isconnected to a robot 48 having a needle driving assembly 52 and needleguide 56 for controlling movement of a needle 60. The needle 60 is usedto deliver therapy to a prostate 64 of a patient. The robot 48 receivesneedle control commands from and transmits needle position informationto the computer 40.

The TRUS transducer 24 is operable to continuously capture radial 2D USimages over a radial operational scan range. The MCM 44 which controlsthe TRUS transducer 24 is in communication with the computer 40 toreceive TRUS control commands via the serial port of the computer 40.The TRUS control commands direct the MOM 44 to control the motorassembly 28. In turn, the motor assembly 28 controls the longitudinalmovement and rotation of the TRUS transducer 24. Additionally, the TRUScontrol commands control the timing of image data capture of the TRUStransducer 24.

The needle driving assembly 52 includes a robotic arm with sixdegrees-of-freedom. The degrees-of-freedom correspond to translations ofthe needle 60 in three dimensions and rotation of the needle 60 aboutthree orthogonal axes. In this manner, the needle 60 can be positionedin a wide variety of orientations. The needle guide 56 is a one-holedtemplate that is used to stabilize lateral movement of the needle 60during insertion.

The computer 40 is a personal computer having a processor that executessoftware for performing 3D image acquisition, reconstruction anddisplay. The processor also executes software for determining dosimetryof a selected therapy, and for controlling the TRUS transducer 24 andthe robot 48. The software executed by the processor includes TRUScontroller software, positioning software, imaging software, 3Dvisualization software and dose planning software.

The TRUS controller software generates TRUS control commands fordirecting the MCM 44, thereby controlling the longitudinal androtational movement and the image data acquisition timing of the TRUStransducer 24.

The positioning software generates needle control commands to controlmovement of the needle driving assembly 52 of the robot 48. Thepositioning software can direct the robotic arm to move in terms ofworld or tool coordinate systems. The world coordinate system is fixedto the ground, whereas the tool coordinate system is fixed to therobotic arm. Further, the positioning software can direct the needledriving assembly 52 to control the longitudinal movement of the needle60.

The imaging software captures, analyzes and processes ultrasound imagesusing the image data retrieved from the memory of the videoframe-grabber 36. The positioning software provides needle positioninformation using the selected coordinate system. In turn, the imagingsoftware directs the TRUS controller software to vary the operation ofthe TRUS transducer 24 as will be explained.

The 3D visualization software renders 3D images to be presented on adisplay (not shown) of the computer 40 using the image data captured andprocessed by the imaging software. In particular, the 3D visualizationsoftware generates three orthogonal views of the target volume: two thatare co-planar to the needle 60 and a third that generally bisects theneedle 60.

The dose planning software performs precise image-based needletrajectory planning. In addition, the dose planning software providesplanned needle trajectory information to the 3D visualization softwareso that the planned needle trajectory can be overlaid atop the US imageson the display. The actual needle trajectory can then be viewed inrelation to the planned needle trajectory. The dose planning softwarecan also receive and process the US images from the imaging software anddynamically re-determine the dosimetry based on the actual needletrajectory and seed implantation locations.

Prior to use, the positioning software controlling movement of the robot48, the needle driving assembly 52 and, thus, the needle 60, and theimaging software are calibrated. During calibration, the mapping betweenthe selected coordinate system of the positioning software and the 3DTRUS image coordinate system is determined and synchronized. In thismanner, the imaging software can be made aware of the expected positionof the needle 60 before detection via imaging.

By unifying the robot 48, the TRUS transducer 24 and the 3D TRUS imagecoordinate systems, the position of the template hole of the needleguide 56 can be accurately related to the 3D TRUS image coordinatesystem, allowing accurate and consistent insertion of the needle via thehole into a targeted position in a prostate along various trajectoriesincluding oblique ones. Further, the operation of the TRUS transducer 24can be varied to focus its attention on the expected position of theneedle 60.

FIG. 2 shows the 3D TRUS transducer 24 capturing a set of 2D US images.As the TRUS transducer 24 is rotated by the MCM 44, it captures imagedata to generate a series of 2D images 68. The 2D images 68 are capturedat generally regular intervals during rotation of the TRUS transducer24. Initially, the TRUS transducer 24 captures a 2D image 68 every onedegree of rotation and rotates through 100 degrees, thereby capturingone hundred and one 2D images 68. The captured 2D images 68 are fannedradially in relation to the TRUS transducer 24. The needle 60 is shownhaving an oblique trajectory in relation to the 2D images 68, andintersects two or more of the 2D images 68.

As will be understood, insertion of the needle 60 along an obliquetrajectory results in the intersection of the 2D TRUS image planes. As aresult, the needle 60 only appears as a point in the captured 2D USimages.

A near real-time method 100 for identification, segmentation andtracking of needles will now be described with reference to FIG. 3. Themethod 100 enables the tracking of the needle 60 even if the needle 60is not coplanar and, thus, exits a 2D US image plane as a result of anoblique insertion. The method can also be used for the identification,segmentation and tracking of needles if they are completely contained ina 2D US image plane. To perform near real-time needle segmentation foran oblique trajectory, capture of two 3D US images is required. A 3D USimage is comprised of two or more 2D US images that are offset. Note,that if the needle 60 is coplanar with a 2D US image, then two 2D USimages can generally be used, but the procedure is unchanged.

The initial 3D US image is obtained by scanning the prostate (tissue) toobtain a set of 2D US images before the needle is inserted. This 3D USimage establishes a baseline or control against which other images willbe compared. A subsequent 3D US image is then acquired by scanning onlythe region containing the needle. It is to be understood that the second3D US image may not be, in fact, the next 3D US image captured after thefirst, but refers to any subsequently-captured 3D US image. The method,as described, is used to identify, segment and track the needle in eachsubsequent 3D US image captured after the first 3D US image is captured.Each new 3D US image is compared to the initial image to identify theposition of the needle at that time.

The method 100 commences with the performance of an initial 3D US scan(step 104). The needle 60 is then inserted into the target volume (step108). Next; a subsequent 3D US scan is performed (step 112). The needle60 is segmented to distinguish its location using the initial andsubsequent 3D US images (step 116). The needle trajectory is thendetermined (step 120). Once the needle trajectory has been determined,the needle tip and needle entry point locations within the reconstructedvolume are determined (step 124). The needle tip and entry pointlocations are then reconstructed (step 128). An arbitrary third point inthe target volume is selected (step 132). The plane defined by theneedle tip and entry points and the arbitrary third point is extractedfrom the reconstructed 3D image (step 136). Next, the extracted plane isdisplayed (step 140). It is then determined if there are any remainingunanalyzed planes (step 144). If there are, the method 100 returns tostep 132, at which another arbitrary point is selected. If, instead, allof the desired planes have been analyzed, the method 100 ends.

During the performance of the initial 3D US scan at step 104, the MCM 44and motor assembly 28 causes the TRUS transducer 24 to rotate about itslong axis over about 100 degrees while image data corresponding to 2D USimages is captured at one degree intervals. The image data correspondingto the 2D US images is then transmitted to the computer 40 to bedigitized by the video frame grabber 36 and registered by the imagingsoftware.

The acquired 2D US images are processed by the imaging software as theyare collected. The 2D US images correspond to planes radially extendingfrom the central axis of rotation of the TRUS transducer 24.Accordingly, the 3D volume is reconstructed by translating and rotatingthe 2D US images with respect to one another. The reconstructed 3Dvolume consists of an array of voxels, or 3D pixels. The voxels aretypically cubic (but can also be rhomboidal) and are arranged accordingto a 3D Cartesian system. Each voxel is assigned a greyscale-level valuebased on the greyscale-level values of the pixels in the translated 2Dimages adjacent to it.

FIG. 4 illustrates a 3D US image reconstructed from the set of 2D USimages. As can be seen, the 3D US image has a fan profile correspondingto the volume imaged by the TRUS transducer 24. The acquired 2D USimages are reconstructed into a 3D US image by the imaging software. The3D visualization software then generates a view of the 3D US image, andprovides a multi-planar 3D display and volume rendering, as well as anextensive set of measurement tools. The 3D US image is then presentedfor viewing on the display of the computer 40. As each new 2D US imageis acquired by the TRUS transducer 24 during its rotation, the 3Dvisualization software dynamically updates the 3D image presented on thedisplay.

During the performance of the subsequent 3D US scan at step 112, aregion of interest is identified, and the ultrasound imaging system 20is focused on a segment of an operational scan range of the TRUStransducer encompassing the region of interest in a target volume. Inparticular, the TRUS transducer is focused on the segment to captureimages of the expected position of the needle 60. While the expectedposition of the needle 60 in the 3D US images can be determined based onthe needle position coordinates provided by the positioning software,needle deviations in the 3D US images can occur for a number of reasons.These include slight bending of the needle 60 as it is inserted andshifting in the target volume. By obtaining a new 3D US image, theactual position of the needle 60 can be more precisely determined.

FIG. 5 better illustrates the performance of the subsequent 3D US scan.The expected needle position is obtained from the positioning software(step 210). The region of interest is determined based on the expectedposition of the needle, and a corresponding segment of the operationalscan range of the TRUS transducer 24 is determined (step 220). Next, ascan strategy for the segment of the operational scan range isdetermined (step 230). In determining the scan strategy for the segmentof the operational scan range at step 230, the positions of 2D US imagesto be acquired is determined. In particular, a set of 2D US images areplanned at one-half degree intervals along the angular width of the scanregion of interest. A scan is then performed in accordance with the scanstrategy (step 240). Data from the initial 3D US image is then used tocomplete the 3D US image (step 250).

During the determination of the region of interest at step 220, theregion of interest is selected to include the expected needle positionobtained during step 210. Where the needle has yet to beinserted/detected, the region of interest is defined to be an areaaround the expected needle entry point. If, instead, the needle was atleast partially inserted/detected at the time of the last 3D US scan,the region of interest is determined to include the original needleposition plus a distance along the needle trajectory beyond the needletip as will be described.

The region of interest is then reverse-mapped onto the operatingcoordinates of the TRUS transducer 24 and is used to determine a segmentof the operational scan range of the TRUS transducer 24 that encompassesthe region of interest at step 230. In particular, the segment of theoperational scan range is selected to correspond to an angular sector ofthe operational scan range of the TRUS transducer 24 that encompassesthe region of interest. Where the needle is inserted along an obliquetrajectory and, consequently, intersects a number of 2D US images atpoints, the angular width of the sector is selected to sufficientlycover the region of interest plus five degrees of rotation to cover thedistance along the needle trajectory beyond the needle tip.

FIG. 6 is an end-view of the TRUS transducer 24 and the segment of theoperational scan range selected during step 220 for the needle when itis inserted along an oblique trajectory. A region of interest 280encompasses an expected needle position 282 and extends a distance pastthe expected needle tip position 284. A segment of the operational scanrange 288 corresponding to the sector encompasses the region of interest280. The segment of the operational scan range 288 includes afive-degree margin 292 to capture the region of interest extending alongthe needle trajectory beyond the expected needle tip position 284. Twobackground areas 296 of the operational scan range of the TRUStransducer 24 flank either side of the sector.

During the completion of the subsequent 3D US image at step 250, datafrom the initial 3D US image is used to fill in the background areas. Asthe scan strategy can exclude the capture of some or all image data fromthe background areas, image data from the initial 3D US scan is used tofill in any image data required in the subsequent 3D US image. The imagedata in the background areas is not expected to change and can, thus, beborrowed from the initial 3D US image.

By modifying the behavior of the TRUS transducer 24 to focus on theregion of interest, more detailed information can be captured around thetip of the needle 60 on a near real-time basis. Further, by reducing thescanning density for the other areas, the additional time required toscan the region of interest can be compensated for.

After the initial and subsequent 3D US scans have been completed, theneedle 60 is segmented at step 116. The subsequent 3D US image iscompared to the initial 3D US image, and the needle position within thesubsequent 3D US image, including the needle tip and entry pointlocation, is determined. The needle 60 will show up as voxels with agreyscale-level change that exceeds a threshold value between theinitial and subsequent 3D US images. There can be, however, other voxelswith a greyscale-level change that exceeds the threshold value that donot, in fact, represent the needle, but may represent, for example,calcifications in the prostate. In order to permit better identificationof the actual needle, the system 20 attempts to identify and discardthese other voxels.

FIG. 7 better illustrates the method of needle segmentation at step 116.The method commences with the calculation of a greyscale-level changethreshold (step 310). A difference map is then generated from theinitial and subsequent 3D US images (step 320). Next, the difference mapis pre-filtered (step 330). Regression analysis is performed on thedifference map to identify the needle (step 340). The result of theregression analysis is then analyzed to determine if it is satisfactory(step 350). If the results are determined to be unsatisfactory, thedifference map is filtered (step 360), and the method returns to step340, where regression analysis is again performed on the filtered image.The filtering of the difference map and the regression analysis isrepeated until all of the voxels in the difference map are within aprescribed range from the regression line. As the filtering removesoutlying voxels, their effect on the linear regression is removed,thereby allowing the needle trajectory to be more accurately estimated.Reiterative filtration of the difference map is performed to obtain adesired level of confidence in the estimated needle trajectory. Once theresult of the regression analysis is deemed to be satisfactory at step350, the method ends.

FIG. 8 better illustrates the calculation of the greyscale-level changethreshold at step 310. A greyscale-level change threshold value, GLCthreshold, is used to reduce the number of voxels to be analyzed in the3D US images and to obtain candidate needle voxels. To determine thethreshold value, the maximum greyscale-level value, in the subsequent 3DUS image is first determined by examining each voxel in the image, andthen GL_(max) is multiplied by a constant.

The calculation of GLC threshold commences with the setting of GL_(max)to zero (step 410). A voxel is then selected from the subsequent 3D USimage (step 420). The greyscale-level value, GL_(value), of the selectedvoxel is determined (step 430). The greyscale-level value of theselected voxel, GL_(value), is then compared to the maximumgreyscale-level value, GL_(max) (step 440). If the greyscale-level valueof the selected voxel, GL_(value), is greater than the maximumgreyscale-level value, GL_(max), the value of GL_(max) is set toGL_(value) (step 450). It is then determined whether there are anyunanalyzed voxels remaining in the subsequent 3D US image (step 460). Ifthere are, the method returns to step 420, where another voxel isselected from the subsequent 3D US image. If, instead, it is determinedat step 460 that there are no remaining unanalyzed voxels in thesubsequent 3D US image, the greyscale-level change threshold value iscalculated as follows:

GLC threshold=a×GL_(max)  (Eq. 1)

where 0<a<1. A value for a of 0.5 provides desirable results.

FIG. 9 better illustrates the generation of a difference map during step320 using the threshold calculated during step 310. The difference mapis a registry of candidate needle voxels that represent an area of thesame size as the initial and subsequent 3D US images. Initially, thegreyscale-level value of each voxel in the initial 3D US image iscompared to that of its counterpart in the subsequent 3D US image, andthe difference is determined:

GLC(i,j,k)=postGL(i,j,k)−preGL(i,j,k)  (Eq. 2)

where preGL(i,j,k) and postGL(i,j,k) are the greyscale-level values ofvoxels at location (i,j,k) in the initial and subsequent 3D US imagesrespectively, and GLC(i,j,k) is the greyscale-level change.

Those voxels in the subsequent 3D US image whose greyscale-level valuesexceed those of their counterpart in the initial 3D US image are deemedto have changed significantly and are registered in the difference map.That is,

(i _(m) ,j _(m) ,k _(m))ε3D DM, where GLC(i _(m) ,j _(m) ,k _(m))>GLCthreshold  (Eq. 3)

for m=1, 2, . . . , n, where n is the number of points included in the3D difference map. The remaining voxels having greyscale-level valuesthat do not exceed those of their counterpart in the initial 3D US imageare deemed to have changed insignificantly and are not added to thedifference map.

The method of generating the difference map begins with the selection ofa voxel in the subsequent 3D US image and its counterpart in the initial3D US image (step 510). The greyscale-level difference, GLdiff, betweenthe voxels of the initial and subsequent 3D US images is found (step520). The greyscale-level difference, GLdiff, is compared to thegreyscale-level change threshold, GLC threshold, to determine if itexceeds it (step 530). If it is determined that the greyscale-leveldifference, GLdiff, exceeds the greyscale-level change threshold, GLCthreshold, the position of the voxel is added to the difference map(step 540). It is then determined whether there are any remainingunanalyzed voxels in the initial and subsequent 3D US images (step 550).If it is determined that there are unanalyzed voxels remaining in theinitial and subsequent 3D US images, the method returns to step 510,where another pair of voxels is selected for analysis. If, instead, itis determined that all of the voxels in the initial and subsequent 3D USimages have been analyzed, the method of generating the difference mapends.

During pre-filtration of the difference map at step 330, voxelsregistered in the difference map are analyzed to remove any voxels thatare deemed to be noise. In the system 20, the 3D image is advantageouslyreconstructed on demand and, therefore, access to the original acquiredimage data is available.

Voxels are identified and analyzed to determine whether they correspondto a characteristic of the needle. Since the image of the needle isexpected to extend along the 3D scanning direction, voxels representingthe needle are assumed to be generally adjacent each other along thisdirection. Other voxels in the difference map that are more than apre-determined distance along this direction from other voxels aredeemed to be noise and removed. That is, assuming that k is thedirection along which the needle is expected to extend, voxels areremoved from the difference map as follows:

$\begin{matrix}{{\left( {i_{m},j_{m},k_{m}} \right) \notin {3{DDM}}},\mspace{11mu} \; {{{where}\mspace{14mu} \bigcup\limits_{m = 1}^{p}{{GLC}\left( {i_{m},j_{m},{k_{m} \pm s}} \right)}} < {{GLC}\mspace{14mu} {threshold}}}} & \left( {{Eq}.\mspace{14mu} 4} \right)\end{matrix}$

where, s=1, 2, . . . , ^(P)/₂, and P is the number of voxels surroundingvoxel (i_(m), j_(m), k_(m)) in the k-direction. A value for P of 4provides desirable results.

FIGS. 10 a and 10 b show the difference map prior to and afterpre-filtration respectively. As can be seen, spurious voxels notoccurring in clusters extending along the same path as the needle areremoved during pre-filtration.

Once the difference map has been pre-filtered, regression analysis isperformed on the difference map at step 340. During this analysis, aline is fit to the voxels in the difference map using linear regressionanalysis. The equation of the line determined from the difference mapusing linear regression analysis provides the estimated trajectory forthe needle.

FIG. 11 better illustrates the performance of the regression analysis onthe difference map at step 340. A voxel registered in the difference mapis selected (step 610). The volume is projected along the z-axis to finda first trajectory (step 620). Next, the volume is projected along they-axis to find a second trajectory (step 630). It is then determined ifthere are any unanalyzed voxels in the difference map (step 640). If itis determined that there are unanalyzed voxels in the difference map,the method returns to step 610, where another voxel is selected in thedifference map for analysis. If, instead, all of the voxels in thedifference map have been analyzed, the results of the first trajectoryare used to obtain y and the results of the second trajectory are usedto obtain z, given x (step 650). Once (x,y,z) has been determined, themethod 240 ends.

If it is determined at step 350 that the linear regression isunsatisfactory, the difference map is filtered at step 360.

FIG. 12 better illustrates the filtering of the difference map. Duringthe filtering of the difference map, spurious voxels that are furtherthan a pre-determined distance from the estimated trajectory of theneedle determined during step 340 are removed.

The method of filtering the difference map commences with the selectionof a voxel in the difference map (step 710). The distance to theestimated needle trajectory is measured in voxels (step 720). Adetermination is then made as to whether the distance between the voxeland the estimated needle trajectory is greater than a pre-determineddistance limit (step 730). It has been found that filtering out voxelsfurther than five voxels in distance from the segmented needletrajectory provides desirable results. If the distance determined isgreater than the pre-determined distance limit, the voxel is removedfrom the difference map (step 740). Then, it is determined if there areany unanalyzed voxels remaining in the difference map (step 750). Ifthere are, the method returns to step 710, wherein another voxel in thedifference map is selected for analysis. If, instead, all of the voxelsin the difference map have been analyzed, the method of filtering thedifference map ends.

FIG. 13 shows the difference map of FIGS. 10 a and 10 b after filtrationat step 360 and immediately prior to the final regression calculation.As can be seen, the difference map is free of spurious voxels distantfrom the visible needle trajectory.

As mentioned previously, once the needle trajectory has been determined,the needle entry point and needle tip locations are reconstructed atstep 124. The needle entry point is determined to be the intersection ofthe needle trajectory and the known entry plane. The needle tip isdeemed to be the furthest needle voxel along the needle trajectory.

After the needle tip and entry point have been reconstructed, anarbitrary third point in the subsequent 3D US image is selected at step128. To extract any plane containing the needle, the segmented needleentry point, needle tip point and a third point within the subsequent 3DUS image are used to define a specific plane that is coplanar with theneedle (i.e., contains the needle lengthwise). The location of thearbitrary point determines whether the plane will be sagital-oblique orcoronal oblique. For a sagital-oblique plane, the arbitrary point ispicked on a line going through the needle entry point and parallel tothe y-axis. For a coronal-oblique plane, the arbitrary point is pickedon a line going through the needle entry point and parallel to thex-axis.

The data occurring along the plane in the 3D US image is extracted atstep 132 to permit generation of a 2D US image of the plane. In thisway, the oblique saggital, coronal and transverse views with the needlehighlighted can be extracted and displayed.

Once the plane is extracted, the 2D US image of the plane is presentedon the display of the computer 40 at step 136. The location of theneedle 60 in the 2D US image is demarcated using a colored line in thegreyscale image to facilitate visual identification of the needle.

It is then determined whether there remain any unanalyzed planes at step140. As three planes are displayed by the computer 40 at the same time,the process is repeated twice to obtain the other two planes. The firstplane selected for analysis is the saggital plane and the other twoplanes are orthogonal to the first plane. If there are, the methodreturns to step 128, where another arbitrary point is selected to defineanother plane. Otherwise, the method 100 ends.

FIGS. 14 a to 14 c show a 2D US image obtained using the method 100during a patient's prostate cryotherapy procedure, demonstrating thatthe needle can be tracked as it is being inserted and orthogonal viewscan be displayed for the user during the insertion procedure.

EVALUATION Experimental Apparatus

The accuracy and variability of the needle segmentation and trackingtechnique was tested using images acquired by scanning phantoms.Referring again to FIG. 1, the robot 48 shown was used to insert theneedle 60 at known angles, including oblique trajectories with respectto the TRUS image plane.

The needle used in these experiments was a typical 18-gauge (i.e., 1.2mm in diameter) prostate brachytherapy needle. The two UStissue-mimicking phantoms were made of agar, using a recipe developed byD. W. Ricky, P. A. Picot, D. C. Christopher, A. Fenster, UltrasoundMedical Biology, 27(8), 1025-1034, 2001, and chicken breast tissues.TRUS images were obtained using an 8558/S side-firing linear arraytransducer with a central frequency of 7.5 MHz, attached to a B-KMedical 2102 Hawk US machine (B-K, Denmark). The computer was a PentiumIII personal computer equipped with a Matrox Meteor II video framegrabber for 30 Hz video image acquisition.

Algorithm Execution Time

Execution time is dependent on the 3D scanning angular interval and theextent of the region to be investigated. To evaluate the execution timeof the disclosed method of needle segmentation the initial 3D US scanwas performed, and then the needle was inserted. After needle insertion,the phantom was scanned again, and the needle was segmented. A softwaretimer was used to measure the time elapsed during the execution of thesegmentation.

Accuracy Test

To test the accuracy of the method, the robot was used to guide theneedle insertion into the phantom at known angles. The angulationaccuracy of the robot was evaluated to be 0.12±0.07 degrees.

First, the robot was used to guide the needle insertion along atrajectory parallel to the TRUS transducer 24, hereinafter referred toas the zero (0) degree orientation. Since the needle could be verifiedby observing the needle in the real-time 2D US image, this trajectorywas assumed to be correct. As a result, oblique trajectory accuracymeasurements could be made with respect to the zero degree trajectory.The positions of the needle tip and the needle entry point were thenfound for the zero degree trajectory using the method described above.The robot 48 was used to insert the needle at different angles (+5, +10,+15, −5, −10 and −15 degrees) with respect to the zero degreetrajectory. For each insertion, the positions of the needle tip and theneedle entry point were found. The corresponding segmented needlevectors through the needle entry point and needle tip were determined byusing the following formula:

$\begin{matrix}{{\cos \; \theta_{a\; 1g}} = \frac{\overset{\rightarrow}{A} \cdot \overset{\rightarrow}{B}}{{\overset{\rightarrow}{A}}{\overset{\rightarrow}{B}}}} & \left( {{Eq}.\mspace{14mu} 5} \right)\end{matrix}$

where {right arrow over (A)} is the segmented needle vector for the zerodegree trajectory; {right arrow over (B)} is the segmented needle vectorfor the insertion at any other angle; θ_(alg) is the angle derived fromthe segmentation algorithm. The accuracy of the algorithm was evaluatedby comparing θ_(alg) with the robot orientation angle θ_(rob). Theerror, ε_(θ), was determined as follows:

ε_(θ)=|θ_(alg)−θ_(rob)|  (Eq. 6)

The accuracy test was repeated with a chicken tissue phantom, and theaccuracy was again determined using Equations 5 and 6. For the agarphantoms, five groups of tests were performed to evaluate the algorithmexecution time and accuracy. Each group consisted of seven insertions;i.e., insertion at 0, +5, +10, +15, −5, −10 and −15 degrees. The meanerror as a function of insertion angle, ε_(θ), was calculated asfollows:

$\begin{matrix}{ɛ_{\theta} = \frac{\sum\limits_{i = 1}^{5}{{\left( \theta_{a\; 1g} \right)_{i} - \left( \theta_{rob} \right)_{i}}}}{5}} & \left( {{Eq}.\mspace{14mu} 7} \right)\end{matrix}$

Results and Conclusion

The following table presents the evaluation results. In the chickentissue phantom, the average execution time was 0.13±0.01 seconds, andthe average angulation error was 0.54±0.16 degrees. In agar phantoms,the average execution time was 0.12±0.01 seconds, and the averageangulation error was 0.58±0.36 degrees. The results shown below alsodemonstrate that the insertion error does not significantly depend oninsertion angle.

Angle (degrees) −15 −10 −5 +5 +10 +15 1 Time 0.13 0.11 0.12 0.12 0.120.14 (seconds) Accuracy 0.50 0.51 0.43 0.37 0.74 0.74 (degrees) 2 Time0.12 0.12 0.12 0.11 0.12 0.13 (seconds) Accuracy 0.30 0.71 0.48 0.680.42 0.86 (degrees

In 3D US images, needle voxels generally have high greyscale-levelvalues. However, due to specular reflection, some background structuresmay also appear to have high greyscale-level values. This increases thedifficulty in automatic needle segmentation in a US image usinggreyscale-level information directly. As US images suffer from lowcontrast, signal loss due to shadowing, refraction and reverberationartifacts, the greyscale-level change detection technique of thedisclosed embodiment of the invention appears to be quite robust. Inaddition, since the needle is segmented from a difference map, complexbackgrounds can be ignored to simplify calculations and accuracy.

In conclusion, a greyscale-level change detection technique has beendeveloped and its feasibility has been tested for near real-time obliqueneedle segmentation to be used in 3D US-guided and robot-aided prostatebrachytherapy. The results show that the segmentation method works wellin agar and chicken tissue phantoms. In addition, the approach has alsobeen tested during several prostate cryotherapy procedures with positiveresults.

Alternative Methods of Defining the Region of Interest and ScanStrategies

A number of alternative methods for defining the region of interest andscan strategies have been explored for use with the system 20. In afirst alternative, the region of interest is defined to include only aset length of the needle from the tip plus a pre-determined distancebeyond the needle tip along the needle trajectory. For example, theregion of interest can be defined to include a one-half-inch length ofthe needle measured from its tip and an area one-half inch along itstrajectory beyond the needle tip. The scan strategy then is selected tocapture 2D US images at one-half degree intervals along the angularwidth of the segment of the operational scan range of the transducer ofthe ultrasound imaging system encompassing the region of interest. Asthe needle is further inserted into the target volume, the region ofinterest roams with the needle tip. Using this approach, 2D US imagescan be rapidly captured and updated to provide accurate informationabout the position of the needle tip.

In another alternative method for defining the region of interest andscan strategy, the region of interest is defined to include an area ofexpected activity of a one-half-inch length of the needle measured fromits tip and an area one-half inch along its trajectory beyond the needletip. This area of expected activity generally allows the new position ofthe needle to be determined when compared to previous images. A scanstrategy can then be selected to scan a segment of the operational scanrange of the transducer of the ultrasound imaging system encompassingthe region of interest using a fine scan density, and other areas usinga coarse scan density. By selecting a relatively high scan density forthe subset of the operational scan range of the transducer of theultrasound imaging system and a relatively low scan density for otherscan areas (e.g. one 2D US image every one-half degree interval in theregion of interest, and every one-and-one-half degree interval outsidethe region of interest), detailed information about the region ofinterest can be obtained while still capturing a desired minimum levelof detail about other areas.

Where the needle has yet to be detected, and information regarding theexpected needle entry point is available, the region of interest can bedefined to include an area surrounding the expected needle entry point.

Where the needle is not determined to be present in the region ofinterest, additional 2D images can be acquired to locate the needle.

Other alternative methods for defining the region of interest and scanstrategy and combinations thereof will occur to those skilled in theart.

While the method of registering the position of an object moving in atarget volume in an ultrasound imaging system and the method of imagingusing an ultrasound imaging system have been described with specificityto a rotational US scanning method, other types of scanning methods willoccur to those of skill in the art. For example, the same approach canbe used with a linear US scanning method. In addition, the segmentationmethod can be applied equally well to 3D US images reconstructed usingthe linear scanning geometry, but acquired using rotational 3D scanninggeometry such as that used in prostate imaging.

The linear regression analysis approach for determining the needletrajectory from the difference map was selected as it requiresrelatively low processing power. A person of skill in the art, however,will appreciate that any method of determining the needle trajectorygiven the difference map can be used. For example, the well-known HoughTransform technique can be employed. The Hough Transform techniquerequires higher computational power than the linear regression approach,but this can be ignored where such processing power is available.

While a specific method of determining the GLC threshold was disclosed,other methods of determining the GLC threshold will occur to thoseskilled in the art. For example, a histogram of the greyscale-levelvalues in the 3D US image can be generated and then analyzed todetermine the regions of the histogram that most likely correspond tothe background and to the needle. The analysis can be based on thestatistical distribution of the greyscale-level values due to theacoustic scattering of the tissue and the statistical distribution ofthe specular reflection of the needle.

In addition to 3D applications, difference maps can be used to registermovement in a single 2D plane. In this case, the difference map couldrepresent a 2D plane and register differences between two 2D images.

While, in the above-described embodiment, the expected needle positionfrom the positioning software was used to determine the region ofinterest thereby to modify the scanning behavior of the TRUS transducer24, one or more previous images could be used to estimate the expectedneedle position. For example, where only the immediately previous imageis available, the region of interest could include the needle plus arelatively large distance along its trajectory beyond the needle tip.Where two previous images are available, the region of interest couldinclude the needle plus a distance along its trajectory beyond theneedle tip, wherein the distance is determined from movement of theneedle registered from the two previous images.

While, in the described embodiment, an object of interest in theultrasound images is a needle, those skilled in the art will appreciatethat the invention can be used in conjunction with other objects, suchas, for example, biopsy apparatus.

It can be advantageous in some cases to compare a US image to one ormore previous US images. For example, where the target volume isexpected to shift, the initial image of the target volume prior toinsertion of the needle may provide an inaccurate baseline image. Byusing more recent previous images, the target volume can be, in somecases, more readily filtered out to generate a cleaner difference map.

While the US images are pre-filtered to identify voxels that areadjacent other voxels along the expected direction that the needlelongitudinally extends, other methods of filtering the images will occurto those skilled in the art. Voxels corresponding to othercharacteristics of an object can be identified to filter out othervoxels that do not correspond to the same.

The above-described embodiments are intended to be examples of thepresent invention and alterations and modifications may be effectedthereto, by those of skill in the art, without departing from the scopeof the invention which is defined solely by the claims appended hereto.

1. A method of registering the position of an object moving in a targetvolume in an ultrasound imaging system, comprising: capturing a firstultrasound image of a target volume; capturing a second ultrasound imageof said target volume; and identifying the position of said object insaid target volume using differences detected between said first andsecond ultrasound images.
 2. The method of claim 1, wherein saididentifying comprises: generating a difference map from said first andsecond ultrasound images identifying said differences therebetween. 3.The method of claim 2, wherein said generating further comprises:thresholding said differences to identify significant changes betweensaid first and second ultrasound images.
 4. The method of claim 1,wherein said first and second ultrasound images are two-dimensional(“2D”).
 5. The method of claim 1, wherein said first and secondultrasound images are three-dimensional (“3D”).
 6. The method of claim5, wherein said identifying comprises: generating a difference map ofsaid differences detected between said first and second ultrasoundimages.
 7. The method of claim 6, wherein said generating comprises:thresholding said differences between said first and second ultrasoundimages to identify significant changes in image voxels.
 8. The method ofclaim 7, wherein said object is a needle.
 9. The method of claim 8,wherein said identifying further comprises: filtering said differencemap to identify voxels corresponding to a characteristic of said needle.10. The method of claim 1, wherein said first ultrasound image iscaptured prior to entry of said object in said target volume.
 11. Themethod of claim 1, wherein said first and second ultrasound images arenot consecutive.
 12. The method of claim 1, further comprising:determining a region of interest in the target volume encompassing atleast a portion of said object; determining a segment of an operationalscan range of a transducer of said ultrasound imaging systemencompassing said region of interest; and focusing said ultrasoundimaging system on said segment of said operational scan range duringimage capture.
 13. An ultrasound imaging system for registering theposition of an object moving in a target volume, comprising: atransducer for capturing a first ultrasound image and a secondultrasound image of a target volume; and a processor for detectingdifferences between said first and second ultrasound images to identifythe position of said object in said target volume.
 14. An ultrasoundimaging system according to claim 13, wherein said processor generates adifference map from said first and second ultrasound images identifyingsaid differences therebetween.
 15. An ultrasound imaging systemaccording to claim 14, wherein said processor thresholds saiddifferences to identify significant changes between said first andsecond ultrasound images.
 16. A method of imaging using an ultrasoundimaging system operable to capture image data from a target volume,comprising: determining a region of interest in the target volume;determining a segment of an operational scan range of a transducer ofsaid ultrasound imaging system encompassing said region of interest; andfocusing said ultrasound imaging system on said segment of saidoperational scan range during image capture.
 17. The method of claim 16,wherein said determining said region of interest comprises: determiningan area of expected activity of an object.
 18. The method of claim 17,wherein said object is a needle.
 19. The method of claim 18, whereinsaid region of interest is determined to correspond to the expectedposition of a tip of said needle.
 20. The method of claim 19, whereinsaid region of interest includes an area along a trajectory of saidneedle beyond said tip.
 21. The method of claim 16, wherein saiddetermining of said region of interest includes the expected position ofa needle in said target volume.
 22. The method of claim 16, wherein saidtransducer is a rotational transducer.
 23. The method of claim 22,wherein said determining of said segment of said operational scan rangecomprises: determining an angular sector of said operational scan rangeof said rotational transducer.
 24. The method of claim 16, wherein saidfocusing comprises: capturing image data in said segment of saidoperational scan range at a greater scan density than outside of saidsegment of said operational scan range.
 25. The method of claim 16,wherein said focusing comprises: capturing image data only in saidsegment of said operational scan range.
 26. An ultrasound imagingsystem, comprising: a transducer for capturing ultrasound images of atarget volume; and a processor for determining a region of interest inthe target volume, for determining a segment of an operational scanrange of said transducer encompassing said region of interest, and fordirecting said transducer to focus on said segment of said operationalscan range.
 27. An ultrasound imaging system according to claim 26,wherein said processor determines an area of expected activity todetermine said region of interest.
 28. An ultrasound imaging systemaccording to claim 26, wherein said transducer is a rotationaltransducer.
 29. An ultrasound imaging system according to claim 28,wherein said processor determines an angular sector of said operationalscan range of said rotational transducer.
 30. An ultrasound imagingsystem according to claim 26, wherein said processor directs saidtransducer to capture image data in said segment of said operationalscan range at a greater scan density than outside of said segment ofsaid operational scan range.
 31. An ultrasound imaging system accordingto claim 26, wherein said processor directs said transducer to captureimage data only in said segment of said operational scan range.